Our Reference: D23-5078363
Sent by email
Email: s22
@rhythmbio.com
Attention: s22
,
Notice under section 41JA of the Therapeutic Goods Act 1989
Requiring information/documents to be provided
Application ID / Submission ID: DV-2022-IVA-11772-1 / DA-2022-04585-1
Sponsor:
Rhythm Biosciences Limited
Manufacturer:
Rhythm Biosciences Limited
GMDN1:
Clinical chemistry tumour marker IVDs [CT845]
Classification:
Class 3
Device Name(s):
ColoSTAT Colorectal Cancer Test System
Sponsor’s Reference:
ColoSTAT
Information is requested by no later than Close of Business 16 March 2023 As a delegate of the Secretary of the Department of Health and Aged Care (the Secretary)
for the purposes of section 41JA of the
Therapeutic Goods Act 1989 (the Act), I have made a
decision to request information in relation to the abovementioned application for the
inclusion of the kind of medical device in the Australian Register of Therapeutic Goods
(ARTG).
I have made this decision because following evaluation of the information provided to the
Therapeutic Goods Administration (TGA) in relation to this application, I am not satisfied
as to all aspects considered in the application’s audit.
For further information, refer to:
1 Information as stated in the application
PO Box 100 Woden ACT 2606 ABN 40 939 406 804
Phone: 1800 020 653 or 02 6232 8644 Fax: 02 6232 8112
Email: xxxx@xxx.xxx.xx
https://www.tga.gov.au
• the relevant legislation:
-
Therapeutic Goods Act 1989 (http://www.legislation.gov.au/Series/C2004A03952); and
-
Therapeutic Goods (Medical Devices) Regulations 2002
(http://www.legislation.gov.au/Series/F2002B00237/Compilations).
The assessment of the associated ColoSTAT Software (application DV-2022-IVA-14630-1)
is currently under review but has highlighted some issues that are common to the
separate applications for each of the two kinds of devices, that are deigned to be used as a
system.
A separate notice will be sent to you, under section 41JA of the Act, for the ColoSTAT
Software. However, the audit of the software is relevant to some information that must be
provided with the ColoSTAT Colorectal Cancer Test System and references made to
information provided for the system, and that is related to the software functionality, will
be referenced in this notice.
1) Manufacturer Relationships
The information that has been provided to the TGA on 12 July 2022 indicates that the
French manufacturer Biotem will manufacture the ColoSTAT Colorectal Cancer Test
System (the Device) on behalf of Rhythm Biosciences Limited.
Clarification is required regarding the relationship between Biotem and Rhythm
Biosciences Limited and to what extent Biotem has been involved in the design and
development of the device and which manufacturer has the design controls for the device.
It is noted that Rhythm Biosciences Limited has stated on 3 December 2020 that ‘the
initial design and broader technology transfer is underway’. It is unclear if Biotem has
participated in any additional design implementation and, if so, to what extent.
Please provide:
• Additional information regarding the relationship between Biotem and Rhythm
Biosciences Limited with particular regard to design controls, roles and
responsibilities.
2) EP 1, EP 2, EP 3, EP 4 and EP 6
The EPs 1, 2, 3, 4 and 6 are respectively related to:
EP1 - Use of medical devices not to compromise health and safety
EP 2 - Design and construction of medical devices to conform with safety principles
EP 3 - Medical devices to be suitable for intended purpose
EP 4 - Long‑term safety
EP 6 - Benefits of medical devices to outweigh any undesirable effects
Page 2 of 30
On 8 June 2022 you were requested to provide detailed information regarding the:
• Risk management report including protocol, report, risk-matrix table, FMEA and
risk benefit statement; and
• Post-Market Data inclusive of all recalls, adverse events, CAPAs, customer
complaints and regulatory refusals.
You have provided Revision 06 of the preliminary risk assessment documentation, signed
6 July 2022. The documentation does not include pre- and post-control measure
implementation matrices and there is no risk versus benefits statement, on the basis of the
consideration of residual risk. The document refers to SOP004 for further detail on
estimation and evaluation of risk. However, this documentation has not been provided.
s47G
You have provided the risk assessment documentation (COL001_164, ColoSTAT Risk
Assessment Report) as evidence of compliance of the Device with the requirements of EP
1, 2, 3,4 and 6 of Part 1 of Schedule 1 of the Regulations. It is noted that the correct
functionality of the software relies, naturally, on the quality of the generated results from
the ELISA testing and the quality of the data input.
The review of the manufacturer’s risk assessment has identified some risks for which the
implementation of control measures requires further discussion, clarification or additional
measures as follows:
Page 3 of 30
s47
In the absence of a control that is prepared in the same manner as the patient samples, for
each Biomarker, there is an unacceptable risk of inappropriate sample dilution that may
not be obvious at the time of data entry (since the quality control material may be in range
but there is no independent verification of the quality assurance of the pre-analytical
sample dilution, using multiple dilution steps, which may exacerbate any pipetting errors).
The manufacturer’s control measure does not appear to be adequate to mitigate the risk of
inappropriate (or accidental) errors in sample preparation and represents an unresolved
risk that lacks an adequate control measure.
Page 4 of 30
s47
s47
. The implemented control measure is
Product Design -
software inputs vetted, Instruction in IFU/Package Insert, and software to flag inappropriate
results.
s47
Hazard 13-1 is with regard to s47
. The control measure to be implemented is:
Appropriate testing introduced to the design process. Therefore, the manufacturer must
verify the expected ranges for a normal and abnormal population for each Biomarker.
Please refer to EP 15 below for a request for additional validation of the Device in this
regard.
Hazards 13-8 and 13-10 are similar and are respectively with regard to
the result from test
is for the incorrect patient and incorrect patient identification from input to result. s47
s47
It is unclear how an error in manual data entry will be detected by the software
verification process if there are no independent software checks for the validity of the data
entry. It is also unclear why the manufacturer has not implemented an electronic
download of results through a health network to avoid the significant risk of error from
manual data entry for the true assayed result/data value and the entry of data under the
incorrect Sample ID for a different patient.
The information provided for the ColoSTAT software (with regard to COL001_235
ColoSTAT Webpage Interface) indicates that the output from the Biomarker ELISA assays,
together with the patient’s demographic information, is manually entered into the Web IU.
The document does not provide information on how the information obtained from the
interaction of the algorithm and the Web UI is then recorded as a physical report.
Additional clarification is required to be provided.
Please provide:
• SOP004 for further detail on estimation and evaluation of risk;
• s47G
• Post-market information including:
o The number of devices supplied since November 2021;
Page 5 of 30
o A complete list of all customer complaints; and
o Details of any corrective and preventative actions, recalls or adverse events
• The final version of the risk assessment for the Device that includes:
o A summation of all risks and hazards from all sources;
o Risk matrices for pre- and post-implementation of control measures; and
o A risk versus benefits statement based on the consideration of residual
risk.
• A discussion regarding the adequacy of the s47
, in the absence of quality assurance for this
step in the analytical process;
• A discussion regarding the s47
. The discussion must include further mitigation of risk, or introduction of
additional control measures, that evaluate assay performance on the basis of
quality assurance procedures that reflect the same analytical process as the
patient samples, to determine assay validity. Clarification is also required with
regard to how Product Design software inputs can possibly determine error if the
error does not result in obviously incorrect results and how an incorrect result
(due to procedural errors) can be identified in the absence of control material(s)
that might potentially identify this error;
• A discussion for the adequacy of s47
;
• A discussion on why the manufacturer has not implemented an electronic
download of s47
;
• Clarification regarding how the information obtained from the interaction of the
algorithm and the Web UI is then recorded as a physical report or electronic
report, for record keeping and traceability;
• Clarification why (if the Web UI is the final physical or electronic report and the
use of the WebUI is encrypted) that the manufacturer considers the s47
Page 6 of 30
3) EP 4 - Long‑term safety and EP 5 - Medical devices not to be adversely
affected by transport or storage
On 8 June 2022 you were requested to provide detailed information regarding the
validation of:
• Shelf-life (3-lots), in-use & transport stability studies
It is noted that the study protocol for real-time and accelerated stability indicates that the
time points for the study span Day 0 to five-years. Since the devices have been released to
market since November 2021 it is expected that the manufacturer should have real-time
stability validation, as of February 2023, of 14-months stability verification to support at
least a one-year expiry.
It is understood that the documentation provided to the TGA in July 2022 is for devices
that have not been stored for this period of time. These comments preface the
requirement for additional and acceptable validation of the real-time stability of the
Device.
The manufacturer has stated that the real-time study will overlap the accelerated stability
validation study, which includes the 5±3 oC storage condition and that, therefore the
protocol for the real-time study is from the 12-month time-point only.
One of the acceptance criteria is identified as: s47
It is also noted that the protocol includes s47
The results of the storage of s47
.
s47
Page 7 of 30
s47
Page 8 of 30
s47
Clarification is required as to whether any of the contents of the Device packaging are re-
usable as some components require disposal after a specified time of opening or
reconstitution. That is, is the supply of a single packaging for the device suitable for use as
one assay or can additional testing be performed once some reagents have been
reconstituted and then discarded after use. The IFU lack clarity in this regard, as it appears
that the number of materials provided indicate a single-use/assay for the device, due to
the limited stability of some of the reagents/reconstituted reagents.
Components of the Device that can be used more than once, once opened, require
validation of in-use stability. You have provided an in-use validation for three-days
stability, expressed on the basis of a change in baseline from T0 for each of the biomarkers.
The acceptance criterion is provided as:
Either the average % difference in concentration from Day 0 of the time point over three
independent experiments is within ±20% OR the average % difference in concentration from
Day 0 of the time point is within ±20% for at least two experiments.
The manufacturer has accepted the performance on the basis of an average of the five test
samples as being <20% variant from the result at T0.
s47
Page 9 of 30
s47
It is not clear if the apparent differences in Device performance represents fundamental
reliability issues with the device or factors causative from reagent changes. The sample
population is not considered adequate to provide statistical identification of the likely
cause of the apparent poor performance and significant variance between measured
results.
The results do not provide evidence that the reliability of the result, based on the use of
stable reagents, has been demonstrated. It is not acceptable to use an averaged value as a
means of accepting and potentially masking the inconsistency of results.
Please provide:
• Replacement real-time stability validation that is inclusive of:
o The full study protocol.
o The sample concentrations for each sample tested for each Biomarker.
o The raw results (assayed and calculated results) for all controls and
specimens for each lot number of device tested and for each biomarker.
o Calculated difference between the T0 and each TX time-point as the actual
difference (U/mL or ng/mL) and %difference.
Note: averaged values are not acceptable; the full data must be provided.
o Control ranges based on the performance and reliability of the Device, with
a maximum 15% CV acceptability.
o Graphed changes to the T0 value as a function of time and within the
acceptance range of ±15%.
• Clarification regarding whether the Device is a single-assay device, based on the
requirement for reconstituted components to be discarded after the initial use
(Detection antibodies, standard and lyophilised control), or whether multiple and
separate components are provided within the Device for multiple assays to be
performed on different days of analysis;
• s47
• Replacement validation data supporting 3-days open vial stability for the Device
for the s47
components of
the Device, when stored at 4 oC and which demonstrates reliable and reproducible
results for the Device (i.e. no extreme variation between results obtained on Day 1
versus the results obtained on Day 3). The study must include the measured
results for each Biomarker at each time point and must include a larger number of
samples spanning the measuring range for each Biomarker to assist in the
identification of error as being stability dependent or independent of stability;
• The stability validation documentation for the s47
standard and the
lyophilised control and the recombinant protein standards (or the identification of
the document provided in the original submission that includes this information);
• Confirmation as to whether temperature data-loggers are to be shipped with each
shipment of devices to Australia (to the end-user) and if this is not the case how
Page 10 of 30
the integrity of the transported device is maintained at 2 – 8 oC during the
transportation period; and
• Any transport stability documentation that demonstrates any impact of the
exposure of the Device to temperatures below 2 oC or greater than 8 oC for an
extended period of time.
4) EP 13 – Information to be provided with medical devices
On 8 June 2022 you were requested to provide:
• Device Labelling, instructions for use and compliance with Regulation 10.2
The requirements for the instructions for use (IFU) that are to be supplied with any
medical device, including IVD medical devices, are provided in Clause (4) of EP 13.
A table of the information that MUST be included in the IFU is provided in sub-clause (3) of
clause (4) of EP 13. Item 29 of this table is specific to IVDs and requires the IFU to be
provided with a device to include the following information:
29 For an IVD medical device, information (including, to the extent practicable, drawings
and diagrams) about the following:
(a) the scientific principle (the ‘test principle’) on which the performance of the IVD medical
device relies;
(b) specimen type, collection, handling and preparation;
(c) reagent description and any limitations (for example, use with a dedicated instrument
only);
(d) assay procedure including calculations and interpretation of results;
(e) interfering substances and their effect on the performance of the assay;
(f) analytical performance characteristics, such as sensitivity, specificity, accuracy and
precision;
(g) clinical performance characteristics, such as sensitivity and specificity;
(h) reference intervals, if appropriate;
(i) any precautions to be taken in relation to substances or materials that present a risk of
infection.
It is understood that the Device is intended to provide an assessment of risk of colorectal
cancer (CRC) and that individual out-put from each of the five component biomarker
assays is not reported, other than within the Web UI interface. It is also understood that
the combination of results obtained using these assays, s47
s47
, provides a qualitative interpretation that is
independent of the quantitation of the measurands that are assayed with the Device.
However, the reliability of the qualitative interpretation depends on the accuracy,
precision and other performance characteristics of the individual biomarkers.
Inappropriate assay result(s) may materially impact the interpretation of the collective
data in-put.
The Device is intended to be used by laboratory health professionals. Laboratories
operating under the requirements of ISO 15189 (which should be most laboratories
within Australia) are required to participate in external quality assurance programs
(EQAP) where possible and if a suitable program is not available for a particular
measurand, alternate procedures must be implemented.
It is noted that the IFU for the Device do not identify the actual Biomarkers that are being
assayed and therefore the mandatory compliance with the requirements of ISO 15189
Page 11 of 30
cannot be demonstrated by the testing laboratory in the absence of the assay identification
for each ELISA.
Additionally, the biomarkers require identification so that the laboratory health
professionals may be aware of any potential endogenous and exogenous sources of
interference, or medical conditions, that are not referenced in the current version of the
IFU, which may decrease or increase a biomarker and which are unrelated to colorectal
cancer.
Without visibility of the analyte being quantitated, the laboratory cannot be aware of any
additional factors that may impact the probability that the algorithm out-put is correct,
and which may result in low or high levels of Biomarker that are not related to the
likelihood of CRC.
On the basis of the information above, the IFU are required to identify each of the five
biomarkers.
s47
The supplementary sheet for the Device includes the ‘Assay Validity – Control Reference
Values for each Biomarker assay. The following statement is included in this
supplementary sheet, in bold type:
Please note that these are reference values only to ensure the validity of the data being
entered into the algorithm.
s47
The supplementary sheet is ambiguous. It appears that the intent is for the control dilution
factor to be applied to the raw results for the controls and not the sample dilution. This is
only an assumption and demonstrates the ambiguity of the supplementary sheet. The
supplementary sheet requires correction to clearly indicate that:
s47
Additionally, it is not clear if the control ranges in the supplementary IFU are fixed ranges
for each and every lot of manufactured control material. Clarification is required as to
whether the supplementary IFU is lot-specific and the control ranges references are
specific for the acceptable control ranges determined for the purposes of batch-release.
The IFU include the statement that:
As a qualitative test, parameters such as linearity are not applicable to the system. Use of the
calibrator serum, and the ELISA precision and accuracy (Section 21) ensures the precision
and accuracy of the qualitative software output.
Although the output of the software algorithm provides a qualitative interpretation, the
five biomarkers provide a quantitative result and linearity is considered to be relevant,
Page 12 of 30
since the accuracy of the data input must be assured to provide the most appropriate
interpretation. Each of the Biomarkers requires the use of a standard curve to calculate
quantitative results for each of the five biomarkers for each sample. As such, the data
output is quantitative. The use of out-of-range results will be discussed later in this notice.
The italicised statement above lacks relevance and is required to be removed. The testing
is quantitative and the test output is indicative (requiring follow-up) rather than
qualitative.
The performance characteristics section of the IFU express linearity in terms of the
dilution. The IFU are required to express linearity in terms of a linear range and must also
include a measuring range, based on the maximal validated pre-dilution that can be
performed in addition to the pre-analytical sample dilution (see below for more
information).
The IFU currently includes statements for each Biomarker that there is no cross reactivity
with other Biomarkers in the assay or members of the same protein value. The IFU must
specify the concentrations of potential cross-reactant or interferent that the manufacturer
has verified as not demonstrating interference or cross-reactivity.
s47
The IFU includes performance characteristics for each Biomarker that present information
regarding the linearity of each Biomarker as a maximum dilution. Since each Biomarker
uses a specific pre-analytical dilution factor, that is standard for all samples, the
information must be presented as a maximal pre-dilution that can be performed, as an
additional dilution, when samples pre-diluted using the standard sample pre-analytical
step are higher than the upper limit of quantitation and require further pre-dilution to
enable quantification.
However, it is noted that the IFU state that when the result is high and reported as ≥x
units/mL (where ‘x’ is the top standard for the relevant Biomarker) the data point is
entered as ‘x units/mL’ (i.e, the highest concentration of the calibration curve). It appears
that the algorithm does not differentiate between results at or above the upper limit of
quantitation and that a high result is adequate for the algorithm to function as intended.
Page 13 of 30
If this is the case it is not clear why any dilution outside the standard pre-analytical
dilution would be a requirement and why the IFU would then reference ANY dilution other
than the predefined dilution. For example, linearity for s47
s47
.
Hazard 13-1 of the manufacturer’s risk assessment is with regard to
Biomarker readings in
the pathology labs are significantly different to those observed in development.
Since the laboratory can only be aware if local expected values for each Biomarker are the
same as the expected values used for device development, through the inclusion of
expected values within the IFU, the IFU are required to include the expected values
(normal ranges) for each of the five Biomarkers -see below (limit of detection).
Please provide:
Replacement IFU that:
• s47
• Identify (supplementary IFU) that the raw results for the controls should be
corrected for the control dilution factor and not the sample dilution factor;
• Clarification if the acceptable reference values for the Biomarker controls in the
supplementary IFU are lot-dependent (with lot-specific supplementary IFU) or if
the control manufacturing process is adequate to ensure that the mean control
target is exactly the same for each lot and for each Biomarker and that the ranges
in the supplementary sheet never change;
• Do not include the statement:
As a qualitative test, parameters such as linearity are
not applicable to the system. Use of the calibrator serum, and the ELISA precision and
accuracy (Section 21) ensures the precision and accuracy of the qualitative software
output;
• Include the linear range, the measuring range and the maximum (validated)
additional pre-dilution of sample that can be performed on the sample in addition
to the standard sample pre-analytical dilution (which varies for each Biomarker);
• Include the concentration of potential interferent or cross-reactant that has been
evaluated, as an upper limit of acceptability (validation);
• s47
•
•
• Provide the maximal pre-dilution factor for each Biomarker that can be used in
addition to the standard pre-analytical dilution factor for each Biomarker. The
information cannot be provided as a combination of the standard pre-analytical
dilution factor and the additional pre-dilution factor for high samples. The
information should also provide an indication that pre-dilution in addition to the
pre-analytical dilution can only be performed when the initial result is outside the
upper limit of the measuring range;
Page 14 of 30
• Clarification regarding the reference to dilutions (linearity) over and above the
standard pre-analytical dilution factor if an additional dilution is never a
requirement (results are used in the algorithm as ‘x units/mL’ if reported as ≥x
units/mL) based on absolute quantitation not being required; and
• s47
5) EP 14 - Clinical Evidence and EP15 - Principles applying to IVD medical
devices only
On 8 June 2022 you were requested to provide detailed information regarding the
validation of:
• s47
• Clinical validation of the algorithm as a predictor of colorectal cancer
You have provided a clinical evidence report (CER) identified as COL001_242 Clinical Trial
Report. The CER makes reference to Appendix D. However, Appendix D appears to be
blank.
It is also noted that the device has been in clinical use since November 2021 and,
therefore, additional clinical information may be able to be provided.
s47
Please provide:
• The raw results for Study 9 (for all samples in Subset A and Subset B) presented as
pre-characterised CRC stages 1,2,3 and 4 and controls sets with the risk
classification for each sample identified as positive likelihood, negative likelihood
or indeterminate;
• Appendix D of the CER; and
• Any additional clinical validation documentation that has been generated since the
release of the device to market. This documentation can be additional studies
performed by the manufacturer or independent reviews of device performance.
Page 15 of 30
6) EP 15- Principles applying to IVD medical devices only
Clauses (1) and (2) of EP 15 state:
1) An IVD medical device must be designed and manufactured in a way in which the
analytical and clinical characteristics support the intended use, based on
appropriate scientific and technical methods.
2) An IVD medical device must be designed in a way that addresses accuracy,
precision, sensitivity, specificity, stability, control of known relevant interference
and measurement of uncertainty, as appropriate.
On 8 June 2022 you were requested to provide detailed information regarding the
validation of the following performance characteristics:
• Specimen Stability and equivalence
• Accuracy
• Repeatability and reliability
• Sensitivity (LoB, LoD & LoQ
• Specificity Studies (including sources of endogenous and exogenous interference
and cross-reactivity
• Evaluation of prozoning or hook effect (if relevant)
• Linearity and measuring range
A) Validation of Sample Stability
Validation of the claimed sample stability of serum samples when stored for up to three
days.
The risk assessment document indicates that Hazard 12-14 is with regard to Biomarker
stability of frozen samples is markedly different to fresh/stored samples that have never
been frozen leading to inadequate training of the algorithm. The control measure requires
appropriate testing introduced to the design process.
Therefore, the manufacturer must verify that the use of frozen samples (for a stipulated
period of freezing), provide equivalent results for samples that are stored at 2 – 8 oC for up
to three days.
Please provide:
• Validation of the stability of the serum sample for storage at three days at 4 oC for
each Biomarker for a range of concentrations (low to high), expressed as
quantified result for each test performed and not as an average % difference; and
• A sample equivalence study that verifies that the use of frozen samples (for a
stipulated period of freezing), provides equivalent results for samples that are
stored at 2 – 8 oC for up to three days.
B) Validation of Accuracy
Validation of the accuracy for each Biomarker assay is based on the use of samples spiked
with each Biomarker. s47
Page 16 of 30
s47
It is also noted that the following comparator devices are available for an evaluation of
device accuracy:
• Schebo M@-PK EDTA Plasma Test (Schebo Biotech AG)
• Human TIMP1 Quantikine ELISA Kit (R&D Systems)
• Human IGFBP2 ELISA Kit (Demedetec)
• Human DKK3 ELISA kit (Ray Biotech)
• Human TBDNF Quantikine ELISA Kit (R&D Systems)
These devices were used for the validation of the stability of the recombinant protein
standards and no justification has been provided as to why these devices could not also be
used for the evaluation of accuracy. s47
s47
Please provide:
• s47
• Validation of the accuracy of each Biomarker assay using the following comparator
devices, which were used in the design validation process, or a rationale why this
was not considered appropriate as a clear and obvious way of demonstrating
accuracy:
o Schebo M@-PK EDTA Plasma Test (Schebo Biotech AG)
o Human TIMP1 Quantikine ELISA Kit (R&D Systems)
o Human IGFBP2 ELISA Kit (Demedetec)
o Human DKK3 ELISA kit (Ray Biotech)
o Human TBDNF Quantikine ELISA Kit (R&D Systems)
• s47
C) Validation of Precision
The evaluation of all potential sources of variance are required to be included in the
validation of precision. It is noted that between-user, between-day, between-instrument
and between-lot precision do not appear to have been performed.
It is also noted that inter-assay precision has been performed over six experiments only.
This is an inadequate number of sequential assays to statistically validate between-run
Page 17 of 30
precision. A minimum of 20-days of testing across three levels of each Biomarker is
required.
The IFU provide information regarding the precision of each Biomarker in the
performance characteristics section. This information is inadequate to represent the
validated (and yet to be validated) precision of the Device.
s47
Clarification as to why the manufacturer considers that samples containing exactly the
same concentration (or the same sample labelled with three identifiers) as being
appropriate for performing the validation of precision is required to be provided.
s47
In the recovery studies the manufacturer has eliminated the results for S5 as being not as
reliable. Validation of precision at concentrations lower than S5 for each Biomarker is
required to be provided.
Please provide:
• Validation of between-user, between-day, between-instrument and between-lot (a
minimum of three lots) precision;
• Validation of inter-assay precision based on a minimum of 20-measurements;
• Clarification as to how it is possible that the exact same concentration (s47
s47 ) for the samples 21RCT076 and 21RCT076 was recorded over six separate
experiments with a calculation based on an optical density which inherently has
natural variance and uncertainty of measurement in any analytical system. Such
precision is extraordinary (and unlikely) for any ELISA-based technology;
• Clarification as to why the manufacturer considers that samples containing exactly
the same concentration (or the same sample labelled with three identifiers) as
being appropriate for performing the validation of precision;
• s47
•
Page 18 of 30
Note: between-day and inter-assay precision can be combined in a single study over 20-
days of analysis. Data for all precision studies must be provided as the measured
concentration for each Biomarker and not as a %CV value.
D) Limit of Detection
The performance validation document provides the acceptance criterion for LoD as:
There
was no specific acceptance criteria set in the validation protocols for sensitivity outside of
that listed in the IFU. Testing to determine the Lower Limit of Quantitation (LLOQ) and Limit
of Detection (LOD) for each assay was performed to provide information on the overall
analytical performance of the assay.
Given that at least one of the Biomarkers is associated with decreased values that (in
association with the other Biomarkers) is associated with risk of CRC, it is considered that
the design requirements for the Device require suitably validated assays that ensure that
the effective limit of the Device at lower analyte concentration is adequately verified. That
is, it is considered that accurate assessment of the true limits of blank (LoB), detection
(LoD) and quantitation (LoQ) is a requirement.
The functional sensitivity of the Device may be a useful tool to determine the reliability of
the Device at lower analyte concentration. To enable an assessment the relationship
between the reliability of the Device at lower levels of Biomarker and a normal population
the manufacture needs to define the normal expected levels for each biomarker as a
normal range study in healthy cancer free individuals.
It is also noted that in the risk assessment documentation (COL001_164, ColoSTAT Risk
Assessment Report) the manufacturer has assessed risks associated with the ColoSTAT
software and factors impacting this software through the use of the ELISA components of
the system.
In particular, Hazard 13-1 is with regard to Biomarker readings in the pathology labs are
significantly different to those observed in development. The control measure to be
implemented is: Appropriate testing introduced to the design process. Therefore, the
manufacturer must verify the expected ranges for a normal and abnormal population for
each Biomarker and include the expected ranges in the IFU so that that the end user may
be aware if the manufacturer’s claimed expected range, for each Biomarker, is in any
material way different to the values expected in the region where the testing occurs.
The validated and claimed LoD values have been compared to the values of each data point
on the calibration curves, as follows:
• s47
•
•
•
Page 19 of 30
• s47
s47
. A clear
rationale for the inclusion of standard values that are likely unreliable is not provided in
any of the manufacturer’s validation documentation. A discussion regarding the likely
poor performance is required to be provided.
Please provide:
• Validation of LoB, LoD, LoQ and functional sensitivity performance that identify
the reliability of the Device at levels below the lower limit of the validated
expected values (normal range) for each Biomarker;
• Validation of the normal (expected) range for each Biomarker and the range of
expected abnormal values indicative of pathological levels of each Biomarker that
is associated with disease, inclusive of CRC; and
• s47
E) Interference
The IFU sates that:
the effects of haemolytic, icteric and lipemic samples have not yet been
evaluated with the ColoSTAT test and such samples should be avoided where possible.
The performance validation studies include an evaluation of the potential impact to the
performance of the Device from haemolysis and lipaemia, but not icterus. IVDs require the
validation of the potential impact of both endogenous and exogenous sources of potential
interference. As the sample type is serum, it is not acceptable to not evaluate the potential
interference from bilirubin as a statement regarding icteric samples being unsuitable is
subjective with regard to the degree of icterus.
The acceptance criterion for the lipaemia and haemolysis study is identified as a recovery
of 80-120% as compared to the reference value for each Biomarker.
s47
This statement is incorrect as the impact from haemolysis for s47
s47
). The study demonstrates that moderately s47
samples and significantly
haemolysed samples are unsuitable for analysis.
It is noted that the manufacturer has tested five different samples for the five biomarkers
three times and taken the average of the results for the five samples and then the average
of the three averages. This is not an acceptable practice.
Repeat interference studies with reported concentrations for each Biomarker in the
presence and absence of potential interferent (haemolgobin, bilirubin and lipid) are
Page 20 of 30
required to be provided. Interference studies for other endogenous substances such as
total protein, paraproteins and rheumatoid factor should also be evaluated.
Please provide:
• Repeat interference studies for haemoglobin, bilirubin, lipid, total protein,
paraprotein and rheumatoid factor with a reported concentration of Biomarker in
a sample with quantitated low, medium and high levels of potential interferent
calculated as a recovery as compared to a quantitated control sample without
interferent;
• Interference studies for exogenous sources of interference that may impact the
Device performance;
• Validation of the risk of carry-over or a justification as to why this is not a
requirement; and
• Validation of any prozoning or a justification as to why this is not a requirement.
F) Cross-Reactivity
The results for the evaluation of potential cross-reactivity with each of the Biomarkers are
expressed as the confidence interval that has been calculated from the s47
s47
of each serum performed from three experiments.
Although the potential cross-reactants did not provide an unacceptable confidence level
(PASS) there is no clarity regarding the actual s47
of each of the
experiments and so it is therefore not possible to determine if low, normal and high levels
of the measurand have appropriately been included within the study.
It is also noted that the acceptance criterion is identified as a 95% confidence interval with
a span of 0±0.2.
s47
It is unclear why the manufacturer has identified the performance as passing the
acceptance criterion when all six of the potential cross-reactants have significantly large
span values. The performance of the s47 assay is in marked contrast to the other four
Biomarkers, for which all potential cross-reactants were within the acceptance criterion.
Please provide:
• Repeat cross-reactivity studies that include the concentration of the measured
result for the sample containing potential interferent and the actual measured
result for the sample not containing potential interferent, with the calculated
difference measure as variance from the true result.
Page 21 of 30
G) Validation of the Measuring Range, Linearity and Parallelism
The IFU indicate that the algorithm requires the following data input with the high
‘decimal’ assumed to be the calculated U/mL (Biomarker 1) or ng/mL (Biomarkers 2-5), as
follows:
s47
This implies that a linear range between these values should be validated to demonstrate
that the data input in the algorithm is based on a true an accurate result.
The standard curve for each Biomarker and the sample dilution factor can be used to
calculate the upper limit of an acceptable data output from the device, as follows:
s47
It is unclear why the quoted high ‘decimal’ values are acceptable, since these exceed the
measuring range for each Biomarker (without additional dilution). It is also unclear why
the acceptable high ‘decimal’ value is close to the upper limit of measurement but is not
the same limit, but rather higher by 4.9% for each Biomarker.
Validation of the high ‘decimal’ as being appropriate (or even possible) for appropriate
data input, is required to be provided.
The manufacturer has assessed the linearity of the Device, for each Biomarker, by using
dilution factors that differ from the standard sample dilution that is presented in the IFU.
The rationale for this practice is not clearly indicated since this represents a departure to
the standardised processing of samples. s47
However, as the s47
s47
The performance validation documentation identifies the ‘nominal’ dilution for each
Biomarker. The nominal dilution is not in alignment with the IFU, as follows:
Page 22 of 30
s47
Clarification regarding the differing nominal dilution and actual dilution values for
Biomarker 1 and Biomarker 3 is required to be provided.
Please provide:
• Validation of the high ‘decimal’ as being appropriate (s47
;
• Clarification regarding the s47
; and
• Repeat linearity/parallelism studies for each Biomarker that are based on the
standard s47
of patient sample followed by s47
to provide a number of data points ranging from the upper limits and lower limits
of quantitation. The information to be provided must include the assayed values.
H) Recovery
In the recovery studies the manufacturer states that:
Additionally, due to the small sample
number (three samples), it was determined that the calculation of standard error was not
suitable for the analysis of the recovery parameter. This result was therefore also removed
from the final calculation.
A recognised flaw in the design of a validation protocol requires an adjustment to the
experimental design to accommodate any detected deficiencies. It is not acceptable to
accept an observed limitation in the design verification phase of product development
without adequate justification.
There does not appear to be any limitation to the manufacturer repeating the experiment
using a larger number of samples for the calculation of standard error.
The recovery reports provide the performance of the Device, for each Biomarker,
referenced as CV% values and not with the additionally required concentration for each
measurement. Since the individual data points for insertion in the algorithm are
quantitative, although the combination of the quantified results and other variable of age
and sex create a qualitative interpretation, recovery is considered relevant and requires
assessment based on reported value.
Page 23 of 30
The current acceptability of recovery is based on average CV%, which may mask the true
performance of the device at different concentrations of Biomarker. s47
Since a decreased level of
Biomarker may be as significant as an elevated level of Biomarker, the recovery at lower
concentrations has some relevance.
s47
For both Biomarker 2 and Biomarker 3 the manufacturer states that for concentrations
equivalent to S5 the performance of the Biomarkers are variable, suggesting that an
evaluation of precision at these concentrations (s47
) is indicated to
evaluate the likely imprecision of the Biomarkers at this concentration.
Please provide:
• Replacement recovery studies that include the theoretical concentration of
Biomarker tested and the assayed result expressed as U/mL or ng/mL and which
include concentrations below S5; and
• If recovery for samples below the concentration of S5 are poor, a discussion on the
recovery in the context of the expected (normal or abnormal concentration of
Biomarker) must be provided.
I) Calibrator and Control Information
Clauses (3) and (4) of EP 15 state:
3) If performance of an IVD medical device depends in whole or part on the use of
calibrators or control materials, the traceability of values assigned to the
calibrators or control material must be assured through a quality management
system.
4) An IVD medical device must, to the extent reasonably practicable, include
provision for the user to verify, at the time of use, that the device will perform as
intended by the manufacturer.
Page 24 of 30
On 8 June 2022 you were requested to provide information regarding:
• Calibrator & Control Information including traceability of the devices to reference
materials or reference performance criteria.
The IFU indicate that when prepared in accordance with the lyophilised control serum
preparation instructions the control will provide validation of the assay if the reported
control result for each Biomarker is within the following range (the mean target value and
standard deviations have been calculated by the TGA on the basis of the low and high
target values provided in the IFU).
s47
It is not clear if each lot of s47
is standardised such that the reportable
values are identical for each lot or if there is variation between target ranges for each lot of
manufactured s47
. If the latter, it is not clear why specific ranges are
specified in the IFU when this information may change between lots. It is possible that lot-
specific supplementary IFU are provided, but this is not clear.
On page 11 of the performance validation report, under 3.1.7 Additional Proteins
(Recovery and Specificity) Details, there is a table listing the specific protein and the
supplier of the protein from whom the protein is sourced. This information is provided
with regard to the performance characteristic studies performed. It is not clear if this
information is also relevant to the proteins incorporated into the manufactured
s47
.
Certificates of analysis for the proteins used in the s47
are required to be
provided.
s47
Whereas the control material will verify if the preparation of s47
s47
are correct (for a single datapoint per Biomarker only) the
Page 25 of 30
control does not serve as a control of the pre-analytical preparation of the patient samples,
which require s47
that are inherently inaccurate, based on the
s47
, especially for Biomarkers 1, 2 and 3.
The use of the reconstituted s47
is not adequate to verify if the
samples have been correctly prepared and that there is no bias or inaccuracy introduced
by poorly calibrated pipettes or poor pipetting technique. Therefore, the control does not
cover the full analytical procedure and does not verify sample preparation. It is also noted
that the use of a single control value for each Biomarker does not provide adequate
coverage of the standard curve to demonstrate appropriate performance across the full
measuring range. Justification for the use of a single control is required to be provided.
Please provide:
• Clarification if each and every lot of s47
has an identical
target mean value and the same acceptance range and, if the information is the
same, how the manufacturing process ensures that the values do not change;
• Clarification if the supplementary IFU is lot specific and, if so, why there does not
appear to be any lot specific information (e.g. lot/batch number and expiry) in the
example provided to the TGA;
• Certificates of analysis for the proteins used in the s47
and
information regarding the sourcing and standardisation of each lot of
manufactured s47
;
• Justification for the use of s47
that does not provide any quality
assurance for the pre-analytical sample preparation, which can intrinsically
provide inappropriate data input into the algorithm without assurance that the
data is appropriate. The justification must include a discussion how the pre-
analytical dilution step can be verified for each Biomarker in the absence of any
direct measure; and
• Justification for the use of a single control value for each Biomarker that cannot
fully evaluate the performance of all measurements across the standard curve, to
the extent possible (in comparison to three levels of quality assurance materials
using low, normal and high control values). The justification must include evidence
that the single control value provides quality assurance for the full measuring
range.
For important information on how to submit your response and your review rights, please
refer to Attachments A and B at the end of this notice.
Yours sincerely
Signed electronically by
s22
Delegate of the Secretary
Medical Devices Authorisation Branch
Therapeutic Goods Administration
13 February 2023
Page 26 of 30
ATTACHMENT A
Important
Please ensure that in your response you provide all information that has been requested.
The first response for this submission will be considered the complete response.
Additional information may be requested to demonstrate compliance with the
requirements of the essential principles. However, failure to provide the requested
information will likely lengthen the audit process timeframe and may adversely impact the
outcome of the audit. You should also be aware that submission of the required
information does not necessarily imply acceptance of the application.
The device application can be ‘pushed back’ in the eBS for you to review if you need to
amend any information provided in or with your device application. If you require any
amendments to be made, send an email to xxxx@xxx.xxx.xx with a request to ‘push your
application back’. The instructions on how to amend the application will be emailed to you
together with the notification when your application is available for you to review. The
request and required changes need to be made by no later than the due date specified in
this notice, and you should inform the Delegate when you have completed the changes.
Timeframe for submitting this information
Regulation 5.2 prescribes that for the purpose of obtaining information that demonstrates
that the matters certified under section 41FD were correct in relation to the compliance
with the essential principles and application of conformity assessment procedures
appropriate to the kind of medical device, the period of obtaining information is
20 working days.
Lapsing or Rejection of the Application
If no information is received in response to this notice to allow the audit of the application,
a decision to vary or not to vary the ARTG inclusion under s9D of the Act will be made
based on the information provided to the TGA.
Withdraw
You may withdraw your application at any time prior to a decision being made to vary or
not to vary the ARTG. You should note, however, that the fee paid for the application is not
refundable (https://www.tga.gov.au/refunds). If you wish to withdraw your application,
you should advise the TGA of this request in writing, via e-mail to xxxx@xxx.xxx.xx
How to present the submission
The requested information must be provided as a complete stand-alone submission.
Cross-referencing to information submitted in support of previous applications that are
already included in the ARTG, or still in process, is not acceptable and will not be
considered or reviewed.
All requested information must be provided in English. Where material is not originally in
English a full translation must be submitted, the accuracy of which is the responsibility of
the sponsor.
All text and pictures must be legible, and pictures must be clearly labelled.
The submission should be sent as an electronic copy (in the form of a CD, DVD or USB
containing all of the relevant material, or as attachments (less than 10 MB in size) via
email to xxxx@xxx.xxx.xx).
For electronic submissions of supporting information larger than 10MB, please email
xxxxxxxxxxxx@xxxxxx.xxx.xx and provide contact details. On receipt of these details, we
shall contact you to arrange registration for our temporary electronic upload facility.
Page 27 of 30
The submitted electronic information must be complete, clearly tabulated, and titled. A
Table of Contents must be included with the submission, clearly identifying all documents
provided in the submission.
Review of the decision
Should you wish to seek a review of my decision to require you to provide
information/documents about the Device, your rights of review are outlined in
Attachment B.
Page 28 of 30
ATTACHMENT B
Request for reconsideration of an initial decision
This decision is a reviewable initial decision under section 60 of the Act. Under section 60,
a person whose interests are affected by a ‘reviewable’ initial decision, can seek
reconsideration of the initial decision.
As this document constitutes written notice of the making of an initial decision being given
by the Secretary, a request for reconsideration of this initial decision must be given to the
Minister in writing within 90 (calendar) days after the initial decision notice is given and
be accompanied by any information that you wish to have considered by the Minister. A
request for reconsideration given to the Minister outside the statutory 90 day
reconsideration period cannot be accepted.
The Minister may either personally undertake a request for reconsideration of an initial
decision or delegate this function to an officer of the Department with the appropriate
delegation.
Under section 60(3A) of the Act, the Minister (or the Minister’s delegate) is not able to
consider any information provided after the making of a request for reconsideration of an
initial decision unless the information is provided in response to a request from the
Minister (or the Minister’s delegate), or it is information that indicates that the quality,
safety or efficacy of the relevant therapeutic goods is unacceptable.
Guidelines for requesting reconsideration of an initial decision
Prior to requesting reconsideration of an initial decision, persons affected by an initial
decision are advised to refer to the TGA website
<https://www.tga.gov.au/reconsideration-reviewable-initial-decisions> for specific
information and detailed guidance for making a request for reconsideration. A request for
reconsideration should then be made in writing, signed and dated by the person
requesting reconsideration and should include the following:
• a copy of the initial decision notification letter, i.e. this letter (or other evidence of
notification);
• identify, and describe with as much specificity as possible, which component(s) of the
initial decision should be reconsidered and set out the reasons why reconsideration is
requested;
• any information/documentation in support of the request, clearly labelled to
correspond with (any or each of) the reasons why reconsideration is requested; and
• an email address nominated for the purposes of receiving correspondence in relation
to the request for reconsideration.
All requests for reconsideration should be given to the Minister by email:
Email:
‘
xxxxxxxx.xxxxxx@xxxxxx.xxx.xx’
Subject:
“
<insert name of person/company making request> - Request for
Reconsideration Under Section 60 of the
Therapeutic Goods Act 1989”
Requests for reconsideration that include material which cannot be attached to a single
email, may be submitted under multiple, sequentially numbered emails (e.g. “… - Email 1
of 3”, “… - Email 2 of 3” etc). All sequentially numbered emails must be given to the
Minister on the same date.
Page 29 of 30
Under section 60 of the Act, the decision upon reconsideration by the Minister (or the
Minister’s delegate) must be to either ‘confirm’, ‘revoke’ or ‘revoke and substitute’ the
initial decision. The Minister (or the Minister’s delegate) must give notice in writing of the
outcome of the decision upon reconsideration to the person whose interests are affected,
within 60 (calendar) days after making a request for reconsideration. If the Minister (or
the Minister’s delegate) fails to give such notice within 60 days, the Minister (or the
Minister’s delegate) is deemed to have confirmed the initial decision.
Subject to the
Administrative Appeals Tribunal Act 1975 (AAT Act), if you are dissatisfied
with the decision upon reconsideration by the Minister (or the Minister’s delegate), you
can apply to the Administrative Appeals Tribunal (AAT) for a review of that decision upon
reconsideration.
NOTE: This initial decision remains in effect unless and until it is revoked or revoked and
substituted by the Minister (or the Minister’s delegate) as a result of a request for
reconsideration under section 60 of the Act OR is set aside, varied or remitted by the AAT
or is otherwise overturned or stayed.
Page 30 of 30
